{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1B54939FD5304D52BE58CA218CD4F991",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "# Python机器学习Kaggle案例实战（第21期） 第2课书面作业\n",
    "\n",
    "## 1. 阅读xgboost相关资料，了解xgboost的背景知识"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E92F947E16954976891F933D9E0ED9C4",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "看了“XGBoost: A Scalable Tree Boosting System”论文以及“Introduction to Boosted Trees”胶片，对于xgboost算法的背景知识理解如下：\n",
    "1. xgboost实际上是多棵决策树(CART树)；\n",
    "2. 在进行算法选择时，采用的目标函数包含了两个部分：$O(x) = \\sum_{i=1}^n l(y_i,\\hat y_i) + \\sum_{k} \\Omega(f_k) $\n",
    "其中$l(y_i,\\hat y_i)$用来表示模型的误差情况，而$\\Omega(f_k)$用来约束模型的复杂程度。\n",
    "在xgboost中针对$l(y_i,\\hat y_i)$项用泰勒展开，展开到2次项（GBDT只展开到1次项）。  \n",
    "针对正则项$\\Omega(f_k)$也是xgboost比较有特色，我理解和线性模型中引入L1,L2有点类似。  \n",
    "在xgboost中$\\Omega(f_k)=\\gamma T+\\frac{1}{2}\\lambda\\sum_{j=1}^{T}w_j^2$，T表示有多少个叶子，以及对所有叶子的权值求和。  \n",
    "3. 在进行迭代计算以确定各个树的参数时，就是遍历各特征，选择一个拆分点，使到目标函数取到最小值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "32655868BC534AE08626836CA4457945",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 2. 数据集Affairs.csv，取自于1969年《今日心理》（Psychology Today）所做 的一个非常有代表性的调查，而Greene（2003）和Fair（1978）都对它进行过分析。该数据从601 个参与者身上收集了9个变量，包括一年来婚外私通的频率以及参与者性别、年龄、婚龄、是否 有小孩、宗教信仰程度（5分制，1分表示反对，5分表示非常信仰）、学历、职业（逆向编号的戈登7种分类），还有对婚姻的自我评分（5分制，1表示非常不幸福，5表示非常幸福）。尝试使用xgboost构建一个分类模型，根据8个自变量判断参与者有婚外情的概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "8828DC850F2B4D0DB424FBA3359C52B7",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "DD5D0E86EAAA4E288EA4F592E79B5484",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_data=pd.read_csv('/home/mw/input/affairs3316/Affairs/Affairs.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "id": "A591EE6BBB3346A78D56D30001AA1CDE",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(601, 10)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "1C001B0696534D329A346D156DC7329A",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>affairs</th>\n",
       "      <th>gender</th>\n",
       "      <th>age</th>\n",
       "      <th>yearsmarried</th>\n",
       "      <th>children</th>\n",
       "      <th>religiousness</th>\n",
       "      <th>education</th>\n",
       "      <th>occupation</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>male</td>\n",
       "      <td>37.0</td>\n",
       "      <td>10.00</td>\n",
       "      <td>no</td>\n",
       "      <td>3</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>female</td>\n",
       "      <td>27.0</td>\n",
       "      <td>4.00</td>\n",
       "      <td>no</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>female</td>\n",
       "      <td>32.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>yes</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>male</td>\n",
       "      <td>57.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>yes</td>\n",
       "      <td>5</td>\n",
       "      <td>18</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>no</td>\n",
       "      <td>2</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  affairs  gender   age  yearsmarried children  religiousness  \\\n",
       "0           4        0    male  37.0         10.00       no              3   \n",
       "1           5        0  female  27.0          4.00       no              4   \n",
       "2          11        0  female  32.0         15.00      yes              1   \n",
       "3          16        0    male  57.0         15.00      yes              5   \n",
       "4          23        0    male  22.0          0.75       no              2   \n",
       "\n",
       "   education  occupation  rating  \n",
       "0         18           7       4  \n",
       "1         14           6       4  \n",
       "2         12           1       4  \n",
       "3         18           6       5  \n",
       "4         17           6       3  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "634ECDE9DD8146D1A0E650FAB2B5991B",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "查看一下有没有缺失值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "BB5785FAA0714A1E83CA5ECD8D714428",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0       0\n",
       "affairs          0\n",
       "gender           0\n",
       "age              0\n",
       "yearsmarried     0\n",
       "children         0\n",
       "religiousness    0\n",
       "education        0\n",
       "occupation       0\n",
       "rating           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_data.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "82CF3B399B7B4E73AF3B9CF364606DEA",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "数据很完整，没有缺失值。下面看看分类型数据情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "5D2F65A13B374CAA821C43D5C99C8BA4",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gender</th>\n",
       "      <th>children</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>601</td>\n",
       "      <td>601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>female</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>315</td>\n",
       "      <td>430</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        gender children\n",
       "count      601      601\n",
       "unique       2        2\n",
       "top     female      yes\n",
       "freq       315      430"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_data.select_dtypes(include=['O']).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B03C19266AE04784AB1A8E5B415EEC1F",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "只有2个分类型变量，同时每个变量只有2个取值，可以考虑用one-hot编码转换。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "8AAEB8FC74AF42A6807167BA76664AD9",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>affairs</th>\n",
       "      <th>age</th>\n",
       "      <th>yearsmarried</th>\n",
       "      <th>religiousness</th>\n",
       "      <th>education</th>\n",
       "      <th>occupation</th>\n",
       "      <th>rating</th>\n",
       "      <th>gender_female</th>\n",
       "      <th>gender_male</th>\n",
       "      <th>children_no</th>\n",
       "      <th>children_yes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>10.00</td>\n",
       "      <td>3</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>4.00</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>5</td>\n",
       "      <td>18</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>2</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   affairs   age  yearsmarried  religiousness  education  occupation  rating  \\\n",
       "0        0  37.0         10.00              3         18           7       4   \n",
       "1        0  27.0          4.00              4         14           6       4   \n",
       "2        0  32.0         15.00              1         12           1       4   \n",
       "3        0  57.0         15.00              5         18           6       5   \n",
       "4        0  22.0          0.75              2         17           6       3   \n",
       "\n",
       "   gender_female  gender_male  children_no  children_yes  \n",
       "0              0            1            1             0  \n",
       "1              1            0            1             0  \n",
       "2              1            0            0             1  \n",
       "3              0            1            0             1  \n",
       "4              0            1            1             0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dummy_feat=pd.get_dummies(df_data[['gender','children']])\n",
    "df_data=pd.concat([df_data,dummy_feat],axis=1)\n",
    "df_data=df_data.drop(columns=['Unnamed: 0','gender','children'])\n",
    "df_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "341809ECCA0F4703A37CADFC04327DBE",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "我们需要构造一个变量用来表示出轨概念，从上表的数据看，affairs=0表示没有出轨，affairs>0表示有出轨。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "A6CAB17E469749B8B6D050F52CE25884",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df_data['affair_rate']=(df_data['affairs']>0).astype('int')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "24AA50963064447584DE95E127955CCD",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>affairs</th>\n",
       "      <th>age</th>\n",
       "      <th>yearsmarried</th>\n",
       "      <th>religiousness</th>\n",
       "      <th>education</th>\n",
       "      <th>occupation</th>\n",
       "      <th>rating</th>\n",
       "      <th>gender_female</th>\n",
       "      <th>gender_male</th>\n",
       "      <th>children_no</th>\n",
       "      <th>children_yes</th>\n",
       "      <th>affair_rate</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>10.00</td>\n",
       "      <td>3</td>\n",
       "      <td>18</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>4.00</td>\n",
       "      <td>4</td>\n",
       "      <td>14</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>57.0</td>\n",
       "      <td>15.00</td>\n",
       "      <td>5</td>\n",
       "      <td>18</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.75</td>\n",
       "      <td>2</td>\n",
       "      <td>17</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   affairs   age  yearsmarried  religiousness  education  occupation  rating  \\\n",
       "0        0  37.0         10.00              3         18           7       4   \n",
       "1        0  27.0          4.00              4         14           6       4   \n",
       "2        0  32.0         15.00              1         12           1       4   \n",
       "3        0  57.0         15.00              5         18           6       5   \n",
       "4        0  22.0          0.75              2         17           6       3   \n",
       "\n",
       "   gender_female  gender_male  children_no  children_yes  affair_rate  \n",
       "0              0            1            1             0            0  \n",
       "1              1            0            1             0            0  \n",
       "2              1            0            0             1            0  \n",
       "3              0            1            0             1            0  \n",
       "4              0            1            1             0            0  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0453A97EEE91461B8EA4608C7EA62C22",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "affair_rate变量为0表示没有出轨，为1表示有出轨。我们将此问题转化为用8个因变量来得出1个自变量的分类问题。\n",
    "现在来拆分一下训练集与测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "63F3F16CFABD463EBA1140EED9F77310",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "C11B4098A98F4BE89F7976A56EC17F0F",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "X=df_data.drop(columns=['affairs','affair_rate'])\n",
    "Y=df_data['affair_rate']\n",
    "seed=1234\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,Y,test_size=0.2,random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "9490F7F6CD6E42579CC64E6FD2D7C663",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "4107ECB4B29C4825812767D318C9E0A8",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,\n",
       "              colsample_bytree=1, gamma=0, learning_rate=0.1, max_delta_step=0,\n",
       "              max_depth=3, min_child_weight=1, missing=None, n_estimators=100,\n",
       "              n_jobs=1, nthread=None, objective='binary:logistic',\n",
       "              random_state=0, reg_alpha=0, reg_lambda=1, scale_pos_weight=1,\n",
       "              seed=None, silent=True, subsample=1)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model=xgb.XGBClassifier()\n",
    "eval_set=[(X_test,y_test)]\n",
    "\n",
    "model.fit(X_train,y_train,early_stopping_rounds=10,eval_metric='logloss',\n",
    "          eval_set=eval_set,verbose=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "ACD14D2981E6411D8265293173C596C1",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_pred=model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "FDDDDB96523443A5B6A7256D05D603BC",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "XGBoost, Accuracy: 76.03%\n"
     ]
    }
   ],
   "source": [
    "predictions = [value for value in y_pred]\n",
    "accuracy = accuracy_score(y_test, predictions)\n",
    "print(\"XGBoost, Accuracy: %.2f%%\" % (accuracy * 100.0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "id": "AFDC2C717E2E4C99B3E2661E8D481BE6",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import roc_curve, auc \n",
    "import matplotlib.pyplot as plt\n",
    "def acu_curve(y,prob):\n",
    "    #  y真实,\n",
    "    #  prob预测\n",
    "    fpr,tpr,threshold = roc_curve(y,prob) ###计算真阳性率(真正率)和假阳性率(假正率)\n",
    "    roc_auc = auc(fpr,tpr) ###计算auc的值\n",
    " \n",
    "    plt.figure()\n",
    "    lw = 2\n",
    "    plt.figure(figsize=(12,10))\n",
    "    plt.plot(fpr, tpr, color='darkorange',\n",
    "             lw=lw, label='ROC curve (AUC = %0.3f)' % roc_auc) ###假正率为横坐标，真正率为纵坐标做曲线\n",
    "    plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\n",
    "    plt.xlim([0.0, 1.0])\n",
    "    plt.ylim([0.0, 1.05])\n",
    "    plt.xlabel('False Positive Rate')\n",
    "    plt.ylabel('True Positive Rate')\n",
    "    plt.title('AUC')\n",
    "    plt.legend(loc=\"lower right\")\n",
    " \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3007F874329E48EBA07EE08EDF70A7CB",
    "jupyter": {},
    "mdEditEnable": false,
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "预测一下婚外情的概率，同时计算一下系统的AUC值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "id": "CD80AA3BEB0141128D6A12863C4DECDD",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"https://cdn.kesci.com/upload/rt/CD80AA3BEB0141128D6A12863C4DECDD/qwd1sxq8f8.png\">"
      ],
      "text/plain": [
       "<Figure size 864x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_pred_proba=model.predict_proba(X_test)\n",
    "acu_curve(y_test,y_pred_proba[:,1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8A18EBF9399C4FBA9D1F2A05621884AE",
    "jupyter": {},
    "notebookId": "60f20e4542cd8a0017864fe3",
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
